The Use of Zoom within Active Vision

Zoom lenses appear to fit very naturally into the framework of active vision — controlling a zoom lens allows an adjustment of the image, enabling either an analysis of a wide scene or a close look at a region or object of particular interest. However, their integration into vision systems is not without difficulty since zoom interacts insidiously with both low and high level processes. This thesis concerns developing and analyzing algorithms that function in spite of zoom; algorithms for visual tracking, camera calibration and Euclidean reconstruction. An approach grounded in visual geometry is adopted, motivated by the notion that the geometric descriptions of point (corner) and line (straight edge) features are zoom-invariant. For tracking, point and line matches in affine views are used to transfer a fixation point into new images in a sequence. Since any affine projection matrix is permitted, the intrinsic camera parameters such as focal length may change freely. If only point matches are available, a previous method based on factorization is applied. When also incorporating lines, the affine trifocal and quadrifocal tensors are used for tracking with zoom in monocular and stereo systems respectively. Methods for the computation of the tensors, minimizing algebraic error, are developed. It is shown that the computation of the affine tensors offer significant advantages in comparison with the projective counterparts in terms of speed and convenience of parameterization. Although dealing with uncalibrated cameras is appealing, calibration information is necessary to set the gains in a visuo-control loop and for extracting Euclidean measurements from the scene. Further contributions of the thesis concern analyzing the reliability of the self-calibration of rotating and zooming cameras under practical situations, motivated by the observation that this configuration of cameras is common for instance in surveillance applications and in sports broadcasts. In particular, degenerate motions are identified and characterized, an analysis of near-ambiguities that may arise when the perspective effects are small is presented, and the effects of violations of the assumption that the motion of the camera is a pure rotation about its optic centre are characterized. Finally, work on self-calibration and Euclidean reconstruction from two or more such cameras is presented. An independent self-calibration of each rotating camera provides a starting point, but this process may be unable to recover all parameters accurately due to couplings between the intrinsic and extrinsic parameters. It is shown that the epipolar geometry may be utilized to alleviate the problem in a collaborative algorithm.